Move2Hear : active audio-visual source separation

We introduce the active audio-visual source separation problem, where an agent must move intelligently in order to better isolate the sounds coming from an object of interest in its environment. The agent hears multiple audio sources simultaneously (e.g., a person speaking down the hall in a noisy household) and must use its eyes and ears to automatically separate out the sounds originating from the target object within a limited time budget. Towards this goal, we introduce a reinforcement learning approach that trains movement policies controlling the agent’s camera and microphone placement over time, guided by the improvement in predicted audio separation quality. We demonstrate our approach in scenarios motivated by both augmented reality (system is already co-located with the target object) and mobile robotics (agent begins arbitrarily far from the target object). Using state-of-the-art realistic audio-visual simulations in 3D environments, we demonstrate our model’s ability to find minimal movement sequences with maximal payoff for audio source separation.Computer Science

Few-Shot Audio-Visual Learning of Environment Acoustics

Room impulse response (RIR) functions capture how the surrounding physical environment transforms the sounds heard by a listener, with implications for various applications in AR, VR, and robotics. Whereas traditional methods to estimate RIRs assume dense geometry and/or sound measurements throughout the environment, we explore how to infer RIRs based on a sparse set of images and echoes observed in the space. Towards that goal, we introduce a transformer-based method that uses self-attention to build a rich acoustic context, then predicts RIRs of arbitrary query source-receiver locations through cross-attention. Additionally, we design a novel training objective that improves the match in the acoustic signature between the RIR predictions and the targets. In experiments using a state-of-the-art audio-visual simulator for 3D environments, we demonstrate that our method successfully generates arbitrary RIRs, outperforming state-of-the-art methods and -- in a major departure from traditional methods -- generalizing to novel environments in a few-shot manner. Project: http://vision.cs.utexas.edu/projects/fs_rir.Comment: Accepted to NeurIPS 202

EVALUATION OF ANTI CANCER POTENTIAL OF METHANOL EXTRACT OF CURCUMA ZEDOARIA.

Objective: Evaluation of anti cancer activity of methanol extracts of Curcuma zedoaria against Ehrlich's ascities carcinoma (EAC) cell line in Swiss albino mice. Method: In vitro cytotoxicity assay has been evaluated by using the trypan blue and MTT assay method. In vivo anti cancer activity was performed by using EAC cells induced mice groups (n=12), at the doses of 100 and 200 mg/kg b.w. respectively, half of the mice were sacrificed and the restwere kept alive for life span parameter. The anti cancer potential of MECZ was assessed by evaluating tumor volume, viable and nonviable tumorcell count, tumor weight, hematological parameters and biochemical estimations. Furthermore, antioxidant parameters were assayed by estimatingliver tissue enzymes. Result: Dose dependent cytotoxicity was observed in (* p < 0.05) Trypan blue and MTT assay method. In vivo anti cancer parameters like tumorvolume, tumor weight, and viable cell count were decreased compared to the EAC control group. MECZ treated EAC cell–bearing mice had anincreased life span to that of EAC control group. Hematological and serum biochemical profiles were restored to normal levels in MECZ-treated micecompared to the EAC control group. Among the tissue parameters lipid peroxidation, reduced glutathione,superoxide dismutase, and catalasetoward normal levels compared to the EAC control group. In short, Conclusion: MECZ exhibited remarkable antitumor activity in Swiss albino mice, which is attributed to its augmentation of endogenous antioxidantactivities and cytotoxic nature. Keywords: Curcuma zedoaria, Zingiberaceae, EAC cell line, antitumor activity, 5-Flurouraci

Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations

Can conversational videos captured from multiple egocentric viewpoints reveal the map of a scene in a cost-efficient way? We seek to answer this question by proposing a new problem: efficiently building the map of a previously unseen 3D environment by exploiting shared information in the egocentric audio-visual observations of participants in a natural conversation. Our hypothesis is that as multiple people ("egos") move in a scene and talk among themselves, they receive rich audio-visual cues that can help uncover the unseen areas of the scene. Given the high cost of continuously processing egocentric visual streams, we further explore how to actively coordinate the sampling of visual information, so as to minimize redundancy and reduce power use. To that end, we present an audio-visual deep reinforcement learning approach that works with our shared scene mapper to selectively turn on the camera to efficiently chart out the space. We evaluate the approach using a state-of-the-art audio-visual simulator for 3D scenes as well as real-world video. Our model outperforms previous state-of-the-art mapping methods, and achieves an excellent cost-accuracy tradeoff. Project: http://vision.cs.utexas.edu/projects/chat2map.Comment: Accepted to CVPR 202